Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GLOBAL_PIP_CACHE setting #3299

Merged
merged 1 commit into from
Nov 23, 2017
Merged

Conversation

humitos
Copy link
Member

@humitos humitos commented Nov 22, 2017

Avoid downloading over and over again the pip packages when you are
developing over a slow internet connection.

Avoid downloading over and over again the pip packages when you are
developing over a slow internet connection.
@agjohnson
Copy link
Contributor

I like it. Does this perhaps do the same thing for you as running a caching pip would? Do you think it's still worth adding a feature, or an example, for using a custom pip server?

@humitos
Copy link
Member Author

humitos commented Nov 22, 2017

I think it's enough with this. I mean, I don't see a real difference at this point between this simple option and configuring a devpi for example. devpi is good when you have multiple users on the same network but for development... This is enough and very simple to set up.

@agjohnson
Copy link
Contributor

👍 I'll do some testing with this setting too

@agjohnson agjohnson merged commit 86f705f into master Nov 23, 2017
@agjohnson agjohnson deleted the humitos/enhancement/global-pip-cache branch November 23, 2017 00:11
kakulukia added a commit to kakulukia/readthedocs.org that referenced this pull request Nov 23, 2017
* 'master' of github.com:rtfd/readthedocs.org:
  Add GLOBAL_PIP_CACHE setting (readthedocs#3299)
  Remove invalid attribute `exception` (readthedocs#3298)
  Update build from database at __exit__ (readthedocs#3292)
  Extra way to check OOM of a command ran inside Docker (readthedocs#3294)
  Add supervisord contrib example (readthedocs#3262)
  Allow promos to be placed in the docs footer (readthedocs#3267)
  Do not convert to bytes the `refresh_token` (readthedocs#3273)
@humitos
Copy link
Member Author

humitos commented Nov 23, 2017

I just noticed that this is not enought to make it work properly.

You can set GLOBAL_PIP_CACHE to whatever you want and everything will continue working but no package will be stored permanently in the disk. Since the command runs inside the container we need to set a path mounted inside the container to make the downloads permanents.

Summarizing, we can just write data permanently only on the directories mounted in the docker container: https://github.com/rtfd/readthedocs.org/blob/2c946d15cb2a43425ff829120e5fc75c19c5a494/readthedocs/doc_builder/environments.py#L639-L652

None of those are good because some are template path and the other is very specific for the current project being built. So, the solution would be mount the self.project.pip_cache_path as well. Like this:

                    self.project.pip_cache_path: {
                        'bind': self.project.pip_cache_path,
                        'mode': 'rw'
                    }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants